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Abstract 

We give a new version of the Shannon-McMillan-Breiman theorem 
in the case of a bijective action. For a finite partition a of a compact 
set X and a measurable action T on X, we denote by C^mai^) the 
element of the partition a V T^q V ... V T'^a V T'^a V .' . .' V T'^a 
which contains a point x. We prove that for /^-almost all x, 

lim ( ^ ] log/z(Cj (x)) = hf,{T,a), 

where fi is a T-ergodic probability measure and h^{T, a) is the metric 
entropy of T with respect to the partition a. 



1 Introduction 

The Shannon-McMillan-Breiman theorem is used in many problems 
related to the metric entropy map of an ergodic measure. We extend this 
well-known result to the case of a bijective dynamical system. Our proof 
follows the line of Petersen's proof [3]. We illustrate this new result with 
an example that gives an inequality between shifts and cellular automata 
entropies and some analog of the Lyapunov exponents. Our bilateral version 
of the Shannon-McMillan-Breiman theorem is expected to be useful in other 
areas of dynamical systems. 



2 Background material 



Let X be a compact space, /i a probability measure on X and T a measurable 
map from X to X. We denote by a a finite partition of X and by C^^{x) 
the element of the partition a V T~^a V ... V T^'^a which contains the point 
X. For all point x the information map / is defined by 

I{a){x) = -log^(Cj(x)) = ^-\ogfi{A)xA{x), 

where Cj(x) is the element of a which contains x and xa is the characteristic 
function defined by 

1 if X e A, 
otherwise. 



Xa{x) 

The information map satisfies 



/(aV/9) = /(a) + /(/?|a), (1) 
I{Ta\T(3) = I{a\(3)oT-\ (2) 

These two properties are easily proved from the definition of / and the fact 
that T is a surjective map. We refer to 01 p. 238], 1^ Chap. 8] for a detailed 
proof of dlBackground materialequation.l] ) and ( 2Background materialequation.l ) 



A simple formulation of the metric entropy with respect to the partition 
a is given by 

hf,{T,a)= lim / /(«| V^=i T'=a)(x) rf^(x), 



E X.n.(.)log(^^ 



where 

Aea,Bel3 

is the conditional information map representing the quantity of information 
given by the partition a knowing the partition P about the point x. 
We recall the Shannon-McMillan-Breiman theorem [2j 

Theorem 1 (Shannon-McMillan-Breiman's theorem) If fi is aT-ergodic 
measure, then for fi-almost all x in a compact X we have 

lim — log/i(Cj (x)) = h^{T,a). 



2 



3 A bilateral version of Shannon-McMillan- 
Breiman's theorem 

In order to prove the main result (Theorem|2)) we need to expose two technical 
lemmas. The proof of Lemma |21 and Theorem |21 requires a bilateral version 
of the BirkhofF pointwise ergodic theorem: for a T-ergodic measure one 
has 

1 " r 

lim f°T'{x)= / f{x)dfi{x) 

n+m^co n + m + 1 ^-^ J X 

k=—rn 

for almost all x with a map / in Li. This result is easily deduced from the 
Birkhoff pointwise ergodic theorem (see |4| Chap. 10]) by breaking up the 
infinite sum in two proportional parts. 

Lemma 1 For all integers m and n we have 

n-l 

I{a\ T-^a) oT'' = J(V™ _„r^a) - I{T-^a). 

k=—m 



Proof. Note that /(V^_„T^a) = /(vg^^T^-^a). Using JTBackground materialequatioiU ) 
we have 

and from ^Background materialequation.l] ) we get 

/(r™a| V™+" r™-^a) = I{a\ T'^a) o T~™. 

Hence, 

/(V™ _„T^a) = liyJ^^T'^-'a) + I{a\ V^+i" T-^a) o T""^. 
The same operations on J(V^"T'"~''a) yields 

= /(V7+i"-^T™-^^^a) + I{a V^+i"-^ T-^a) o T" 

Hence, 



m+1 



a] 



Iterating similarly t — 1 times on /(V™j^" ^T"" ^ ^a) leads to 

t 

J(VJL _„T^a) = /(V™+"-*T'™~*-^a) + ^ I{a\ v}l+"-^ T-^a) o 

fc=0 

Taking t = m + n — 1 gives 

m+n— 1 

/(Vf=_„T^a) = +/(T-"a) + ^ Vf +1""'= T-^a) o T 

fc=0 

which completes the proof. 
□ 

Lemma 2 If fi is a T ergodic measure then for almost all x in X, 



—m+k 



lim - 

m+n^oo m + n + 



^ V lim J(a| T~^a) o T\x 

A;=— m 

^ ra— 1 

lim V I(a\ V"-,^' T--'a) o T'^fx) 

m+n^oo m + n + 1 -^^^ 



A;=— m 

Proof. For notational convenience, we introduce 
/ = lim I{a\ V^^i T-^a) and Fn = sup |/(a| V^^i T-^a) - f\. 

It is well known that the sequence '^^j=i T^^a))sm converge almost 

everywhere and in Li. The proof of this convergence (see |31 Chap. 8] and 
p. 262]) requires the increasing martingale theorem. 
We need to show that 

^ n— 1 

lim V \l{a\ V"-{= o - / o T^l = 0. 

m+n^oo m + n + 1 ^-^ ■' 

k=—m 

Note that 

n-l 



m + n + _ 

k=—m, 



1 ^ 

V |/(a| V"rf T-^a) oT'^ - /oT'^l < 

n + l I ^ I j-i ^ I - 

n— 1 

V |/(a| v^rf T-^a)or^-/or*^| 

fc=n-Af 
n-AT-l 

n + 1 



/e=— m 
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If we fix and let n + m tend to infinity then the first term in the right-hand 
side of the above inequahty goes to zero. Since the map F/v belongs to Li 
(see PI), the bilateral version of Birkhoff's ergodic theorem applies and we 
can assert that 

lim V |Fjv|oT^ = / FNd/j.. 

m+n^oo m + n + 1 ^-^ Jx 

k=—rn 

Since limTv^oo = 0, the dominated convergence theorem implies that 
F/v dfi tends to zero which completes the proof. 
□ 

Theorem 2 For a bijective map T from X to X and a T-ergodic measure 
11, we have for ii-almost all x 

lim log/i(Cj (x)) = h^(T, a), 

where Cj^ ^(a:) represents the element of the partition a V Ta ... V T'^a V 
T^^a V ... V T~'^a containing the point x. 

Proof. Since the sequence (/(a;| V^^^ T~^a)) ^^^^ converges to a Li map 
and by using the dominated convergence theorem, it follows that 



/i^(r,a) = lim / I{a\y'^-^T-^a){x)dJx) = / lira I{a\y'^^T-^ a){x) dJx) 
Jx Jx 

for y[x-almost all x. The bilateral version of Birkhoff's ergodic theorem implies 
that for almost all x 

^ n— 1 

hJT, a) = lim V lim I(a\ T-^a) o T\x). 

k=—m 

From Lemma El it follows that 



n— 1 

h,{T, a) = hm Yl Vi=i' o T" 

m+ri-»oo m + n + 1 — ^ ■' 



m+n— »oo m + n + _ 

k=—m 



\x . 



Using Lemma C] for almost all x we obtain 

h,{T,a)= hm ^ {l{y]L_^T^a){x)-Iin){x)). 

5 



Since {I{T "a))„gN is bounded for /i-almost all point x, the sequence ^^^^^,^1 
tends almost surely to zero. Hence, 

h,{T,a) = lim J I{y]l_^T^a){x) 
m+n^oo m -\- n 



4 An illustration 

In this example we do not give a definition of the particular discrets dy- 
namical systems called cellular automata; the reader can find a survey in 
and the complete proof of this illustration in j5j. The bilateral version of 
the Shannon-McMillan-Breiman theorem is needed to establish an inequal- 
ity between the entropy of a cellular automaton F denoted by a), the 
entropy hfj,{a) of a particular bijective cellular automaton a called the shift, 
and some discret analog of the Lyapunov exponents. Here the measure /i is 
F-invariant and cr-ergodic. 

The standart Shannon-McMillan-Breiman theorem ^ Chap. 10] says that 
in the case of an invariant measure fi. 

h,{F,a)= [ lim ziiog^(C„^Ja;)). 

In jHj one proves that there exists some integer and bounded maps /„ and Qn 
such that, for all point x, one has 

with lim^^oo .fn{.x) + <7„(x) = +oo for /i-almost all point x for a certain class 
of cellular automata. For those that do not belong to this class, the entropy 
is equal to zero (see [Sj). With these properties we obtain 

K{F,a) < y Jim -^log/i (CJ^(,),,„(,),Jx)) dfi{x) 

and 

^9n{x) + Ux) + l 

n 
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The bilateral version of the Shannon- McMillan- Breiman theorem implies that 

fn{x) + 9n{x) + 1 



n 



hn{F,a) < hij,{a,a) / liminf ■ 

J n— >oo 

Using the Fatou lemma, we have 

h^{F, a) < h^{a, a) x (A+ + A;), 

where 

/fn ( ^ ) f Qtl ( ^ ) 
— duix) and = liminf / — daix) 
n ^ J n 

are called the left and right average Lyapunov exponents. 
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